Imaging method and device

ABSTRACT

Disclosed are an imaging method and device. The method includes: acquiring, at a specified imaging time, a pulse sequence in a time period before the specified imaging time, with regard to each pixel of a plurality of pixels; calculating a pixel value of the pixel according to the pulse sequence; and obtaining an image at the specified imaging time according to a space arrangement of the pixels, in accordance with pixel values of the plurality of pixels at the specified imaging time.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is continuation of U.S. application Ser. No.16/724,906, filed on Dec. 23, 2019, which is a continuation-in-part ofU.S. application Ser. No. 16/042,225, filed on Jul. 23, 2018, acontinuation-in-part of U.S. application Ser. No. 16/666,969, filed onOct. 29, 2019, and claims priority to Chinese Patent Application No.201910027914.X, filed on Jan. 11, 2019. The U.S. application Ser. No.16/042,225 is a continuation of International Patent Application No.PCT/CN2017/072038 filed on Jan. 22, 2017, which claims priority toChinese Patent Application No. 201610045011.0, filed on Jan. 22, 2016.The U.S. application Ser. No. 16/666,969 claims priority to ChinesePatent Application No. 201910027914.X, filed on Jan. 11, 2019. Allcontents of the aforementioned applications are incorporated byreference herein.

TECHNICAL FIELD

The present disclosure relates to the field of information codingtechnology, particularly to an imaging method and device.

BACKGROUND

The form that exists in a certain time and space is a source ofinformation, for example, the propagation of light in space formsdynamic images, the flow of large numbers of water molecules producesocean information, and the dynamic movement of air molecules and otherfloats forms climate information. In terms of dynamic images, humans andcreatures perceive the world by capturing photons through their eyes,and the modern camera records the dynamically changing world by usingCCD (Charge-coupled Device) or CMOS (Complementary Metal OxideSemiconductor) to capture photons, which generates large amounts ofimage and video data.

Traditional methods of representing dynamic images are two-dimensionalimages and videos as sequences of images. The traditional image is atwo-dimensional information form. The narrowly defined image is theresult that light is projected on a photographic plane after reflection,diffuse reflection, refraction or scattering in the physical world. Thegeneralized image includes any information form distributed on atwo-dimensional plane. The image represented by the digital form is moreconvenient to process, transmit and store, so it is necessary totransform an image in the analog signal form into an image representedby the digital form, i.e. a digital image. The process of imagedigitization mainly includes three steps: sampling, quantization andcoding. Sampling is a process of discretizing the space distribution ofan image. For a two-dimensional image, the most common way is to dividea rectangular area covered by the image into equal-sized sampling pointsat equal intervals, and the number of rows of sampling points and thenumber of sampling points per row are usually called digital imageresolution (more accurate resolution refers to the number of samplingpoints per unit physical size). Quantization is a process ofdiscretizing the color (or other physical quantities) of an image ateach sampling point, which is generally represented by a quantizationlevel. The quantized values of each sampling point and its color (orother physical quantities) form one pixel of the image, and all pixelsarranged in rows or columns form a digital image.

The traditional concept of video is a sequence of images obtained at acertain time interval. An image in the sequence is also called a frameimage. Therefore, a video is also an image sequence. The time intervaldivision between images is also a part of sampling. Usually, equalinterval division is adopted, and the number of images collected persecond is called frame rate. In order to ensure that the information isnot lost in the process of digitalization, that is, complete restorationcan be achieved when the information is restored to the analog form,according to the sampling theorem, it needs to be sampled at least twicethe frequency of the image spatial signal.

The video collected in the traditional way produces a large amount ofdata after the digitalization. Taking a high-definition video as anexample, the amount of data per second is 1920×1080×24 bits×30 framesper second=1492992000 bits per second, which is about 1.5 Gbps. It isalmost impossible for network and storage technology to transmit suchamount of data through a broadcast communication network, or to providevideo services for thousands of users on the internet, or to store videodata generated by millions of cameras in cities for 24 hours. A largeamount of redundancy needs to be removed in high-precision digital videodata, which is the central goal of digital video coding, so digitalvideo coding is also called digital video compression. From the researchof Hoffman coding and differential pulse coding modulation in the late1840s and early 1850s, video coding technology has experienced thedevelopment for nearly 60 years. In this process, three types ofclassical techniques including transform coding, prediction coding andentropy coding were generally formed to remove spatial redundancy,temporal redundancy and information entropy redundancy of video signalsrespectively.

Based on the requirement of technology accumulation and informationtechnology development for more than 30 years, various video codingtechnologies began to converge in the 1980s, and gradually formed ablock-based hybrid coding framework of prediction and transformation.The hybrid coding framework was standardized by the standardizationorganization, and began to be applied on a large scale in the industry.There are two major international organizations specializing in theformulation of video coding standards in the world, namely the MPEG(Motion Picture Experts Group) organization under the ISO/IEC and theVCEG (Video Coding Experts Group) organization of the ITU-T. The MPEGfounded in 1986 is specifically responsible for the developing ofrelated standards in the multimedia field, which is mainly used instorage, broadcast television, streaming media on the Internet orwireless network and so on. ITU, the International TelecommunicationUnion, mainly formulates video coding standards for the field ofreal-time video communications, such as video telephony, videoconference and other applications. The AVS working group, set up byChina in June 2002, is responsible for formulating corresponding digitalaudio and video coding standards for the domestic multimedia industry.

In 1992, the MPEG organization formulated the MPEG-1 standard (launchedin 1988, was a superset of ITU H.261) for VCD (Video Compact Disk)application with a data rate of about 1.5 Mbps; in 1994, the MPEG-2standard (launched in 1990) for applications such as DVD and digitalvideo broadcasting was released, which is applicable to bit rates of1.5-60 Mbps or even higher; in 1998, the MPEG formulated the MPEG-4standard (launched in 1993, based on the MPEG-2 and H.263) for low bitrate transmission. ITU basically kept pace with the development of theMPEG, and also formulated a series of H.26x standards. The H.261standard, which began in 1984, was a precursor to the MPEG-1 standardand was basically completed in 1989, mainly formulated for realizingvideophone and video conference on ISDN. On the basis of H.261, theITU-T formulated the H.263 coding standard (launched in 1992) in 1996,and successively introduced H.263+, H.263++, etc.

In 2001, the ITU-T and the MPEG jointly established the JVT (Joint VideoTeam) working group, and set up a new video coding standard. The firstedition was completed in 2003. The standard was called the tenth part ofthe MPEG-4 standard (MPEG-4 PartAVC) in the ISO, and called the H.264standard in the ITU. Four months later, the Microsoft-led VC-1 videocoding standard was promulgated as an industry standard by the Societyof Motion Picture and Television Engineers (SMPTE) of America. In 2004,a national standard with independent intellectual property rights wasdeveloped in China, and it was promulgated as a national standard of“Information Technology Advanced Audio and Video Coding Part II Video”(National label GB/T 20090.2-2006, usually referred to as the AVS videocoding standard for short) in February 2006, after industrializationverification such as chip implementation. These three standards areusually referred to as the second generation video coding standard, andtheir coding efficiency is double that of the first generation, and thecompression ratio is up to about 150 times, that is, a high-definitionvideo (under the condition that the quality meets the broadcastrequirements) may be compressed to 10 Mbps or less.

In the first half of 2013, ITU-T H.265 and ISO/IEC HEVC (High EfficiencyVideo Coding) as the third generation video coding internationalstandard were promulgated, and the coding efficiency was doubled that ofH.264. In parallel with this, China formulated the second generation AVSstandard AVS2, which is called “Information Technology EfficientMultimedia Coding”. Compared with the first generation AVS standard, thecode rate of AVS2 is reduced by more than 50%, which means that thecoding efficiency is doubled. For a scene-like video such as amonitoring video, the compression efficiency of AVS2 is further doubled,and up to four times that of AVC/H.264, that is, the compressionefficiency has reached 600 times.

Although modern video coding technology has already achieved remarkableresults and has been widely applied, and the compression efficiency hasrealized “doubling every ten years”, it is far from reaching an ideallevel. According to the existing research report, the global data volumereached 2.84 ZB in 2012. By 2020 the figure will rise to 40 ZB, whichwill double about every two years, of which the monitoring video willaccount for 44%. In other data such as health data, transaction data,network media, video entertainment data, etc., the image and the videowill also account for a large proportion. In China, more than 30 millioncameras have been installed in public places, and these cameras haveproduced nearly 100 EB video which requires hundreds of billions of yuanin storage. Therefore, the technological progress of “doubling every tenyears” in video coding efficiency has been far from satisfying the rapidgrowth of “doubling every two years” in video big data, and how toimprove the video coding efficiency has become a major challenge in theinformation age.

As mentioned above, the formation of video concept originates from thedisclosure of film, and the basis for the technique scheme ofrepresenting a video with an image sequence is visual persistencephenomenon of human vision. The film uses 24 frames per second and thetelevision uses 25 or 30 frames per second, which can basically meet theneeds of human eyes to get a continuous sense. This technical setting isalso solidified as a technical formula with the wide application offilm, television and personal camera equipment. However, thedisadvantages of this method of representing dynamic images are alsoobvious. It can't record high-speed movements such as a rotating wheel,a high-speed sport table tennis or even soccer. It also fails to catchthe movement details in video monitoring, and it can't supportscientific research, high-precision detection and other specialrequirements. New high-definition and ultra-high definition televisionsare also trying to increase the frame rate to 60 frames per second oreven higher to better represent high-speed sports such as table tennis.However, such a video frame rate can't represent a faster changingphysical phenomenon, so high frequency cameras appear. Their frame ratecan reach 1000 frames per second, even 10,000 frames or higher. Theproblem is the large-scale growth of data volume, and the correspondingacquisition and processing circuit design are expensive or evenimpossible. More importantly, the increase in the frame rate meansexposure time of a single frame is reduced, and the exposure of thecollected single frame image is seriously insufficient. A way tocompensate this is to increase pixel size, which brings about thereduction of spatial resolution. In the final analysis, all of theseproblems are caused by video acquisition and representation using “firstspace, after time” equal time interval method. This method is only atechnological choice based on the persistence characteristics of humanvision when the film appears, does not mean it is the best solution torepresent dynamic images.

Therefore, it is an urgent problem to develop an effective video codingmethod that takes account of temporal information and spatialinformation simultaneously.

Machine vision algorithms based on traditional image sensors have beenwidely used in many fields such as intelligent safeguard and intelligenttransportation. However, the traditional image sensors becomeincreasingly incapable of meeting current visual mission requirementsdue to design concepts thereof. These sensors generally perform acomplete sampling on a scenario at a preset fixed frequency and in aunit of frame. Such sampling based on fixed frame rate cannot reflect adynamic change of the scenario, and is prone to an oversampling orundersampling on the current scenario, thereby causing problems such asa large redundancy of video data, a low time domain resolution, andblurring under high speed motion. Inspired by a visual samplingmechanism of biological retina, new types of cameras that collect spikearray signals are developed, including sensors that fire spike signalsbased on a change in illumination intensity, such as Dynamic VisionSensor (DVS), Asynchronous Time-based Image Sensor (ATIS) and Dynamicand Active Pixel Vision Sensor (DAVIS), and sensors that fire signalsbased on accumulative illumination intensity, such as illuminationintensity accumulation sensors. Sensors of such type of camerascollecting information of optical signals in a certain region during acertain period of time, and have advantages of high dynamic range, hightime resolution, and the like.

Transforming signals into human-viewable display forms is the first stepin intuitively understanding spike signals. Since most of the existingmachine learning algorithms cannot directly process spike signals asdata sources, it is desired to provide a technology capable ofvisualizing spike signals.

SUMMARY

In order to solve the above mentioned problems, the present disclosureadopts the following technical solutions.

According to one aspect of the present disclosure, a method for encodingspace-time signals is provided, including:

collecting a space-time signal at each one of local spatial positions ina monitoring area, the local spatial positions constituting themonitoring area;

performing time-domain coding on the space-time signal at the each localspatial position to obtain a pulse sequence representing a changeprocess of the space-time signal at the each local spatial position; and

performing space-domain coding on pulse sequences of all of the localspatial position according to a spatial position relation to obtain apulse sequence array.

Performing time-domain coding on the space-time signal at the each localspatial position to obtain a pulse sequence representing a changeprocess of the space-time signal at the each local spatial positioncomprises:

accumulating the space-time signal at the each local spatial positionaccording to time to obtain a cumulative signal intensity value;

transforming the cumulative signal intensity value, and outputting apulse signal when a transformation result exceeds a specific threshold;and

arranging pulse signals corresponding to the each local spatial positioninto a sequence according to time, so as to obtain the pulse sequencerepresenting the change process of the space-time signal at the eachlocal spatial location.

Collecting a space-time signal at each one of local spatial positions ina monitoring area comprises:

collecting the space-time signal from a designated local spatialposition by each signal collector to complete time-domain sampling; andcompleting space-domain sampling of the monitoring area by a pluralityof signal collectors arranging into an array and cooperating with eachother to cover the monitoring area.

The space-time signal is an optical signal, and the signal collector isa photosensitive device, and accumulating the space-time signal at theeach local spatial position according to the time to obtain a cumulativesignal intensity value comprises:

an electrical signal intensity output by an signal collector through aphotoelectric conversion being positively correlated to a collectedlight intensity; the signal collector being connected to one or moresignal accumulators, and the signal collector transmitting theelectrical signal intensity to the connected signal accumulators;

accumulating a signal in a past period of time by the signalaccumulators, and output ends of the signal accumulators being thecumulative signal intensity value; one signal accumulator beingconnected to one filter, and the signal accumulator transmitting thecumulative signal intensity value to the connected filter; and

the filter taking one or more signal accumulators as input, transforminginput cumulative signal intensity values according to a specific filterfunction, and outputting a pulse signal corresponding to a local spatialposition when a transformation result exceeds the specific threshold.

When the signal collector is connected to multiple signal accumulators,the signal collector outputs the electrical signal intensity to alldownstream signal accumulators at a same time; or, the electrical signalintensity is evenly assigned to all downstream signal accumulators; or,the electrical signal intensity is assigned to all downstream signalaccumulators according to a certain weight, and the weight is a functionof a spatial position distance between the multiple signal accumulatorsand the signal collector.

The multiple signal accumulators are time-limited rolling signalaccumulators or time-unlimited signal accumulators, the time-limitedrolling signal accumulators only accumulate a signal in a specificperiod of time before a current time, and an earlier signal isautomatically cleared; and the time-unlimited signal accumulatorsimplement accumulation continuously.

The filter function of the filter is set according to a sparsity of aspatial position that the filter needs to capture, and the sparsity ofthe spatial position that the filter needs to capture is determinedaccording to a local spatial scope of a signal collected by a signalcollector associated with the filter; and/or

the monitoring area is covered by using multiple filters through aredundant design, and a sparse mode of the each local spatial positionin the monitoring area can be captured by a corresponding filter; and/or

a multi-level redundancy design is adopted on space scale coverage forthe multiple filters, the filters of different levels are sensitive tospatial sparsity of corresponding space scales, and an effective captureof any scale sparsity in the monitoring area is realized by acooperation of the multi-level filters; and/or

the filter transforms the cumulative signal intensity values from theone or more signal accumulators according to a set filter function, soas to obtain transformation coefficients related to signal distributionwithin a local spatial position, when a transformation coefficientexceeds a preset threshold, the filter outputs a pulse signal whichcarries a pulse intensity, and there is a corresponding relationshipbetween the pulse intensity and the cumulative signal intensity value;and/or

after the filter outputs pulse signals, all signal accumulatorscorresponding to the filter are reset.

The filter is a binary filter, and the filter function is a thresholdfunction;

when there is only one signal accumulator as an input of the binaryfilter, if the cumulative signal intensity value input by the signalaccumulator exceeds a specified threshold, the binary filter outputs apulse signal, otherwise no pulse signal is output; or,

when there are multiple signal accumulators as inputs of the binaryfilter, the binary filter implements a simple accumulation or a weightedaccumulation according to certain rules, and performs a filteringtransformation for cumulative signal intensity values input by themultiple signal accumulators, if a transformation result exceeds aspecified threshold, the binary filter outputs a pulse signal, otherwiseno pulse signal is output.

The method further comprises:

performing time-domain discrete representation for pulse outputs of thefilter by using a high frequency clock, so that the pulse outputs of thefilter occur only at times of equal intervals, transformationcoefficients output by all filters at a same time forming a sparsearray, a transformation coefficient corresponding to filters withoutoutput at the time being 0, and sparse arrays being arranged into atransformation coefficient array in a sequence of equal time intervals.

Pulses output binary filter are represented by binary numbers, when thebinary filter outputs a pulse, it is represented by 1, otherwise it isrepresented by 0; at a same time, a binary filter with a pulse outputoutputs 1, and a binary filter without pulse output outputs 0; and alloutputs at the same time constitute a binary sparse array according to afilter array, and binary sparse arrays at all times are arranged into abinary sparse sequence array according to a sequence of equal timeintervals specified by a clock.

The method further comprises:

a signal collector, a signal accumulator and a filter constituting a“collection-accumulation-transformation” triplet in a one-to-one form,that is, each signal collector outputs only one signal accumulator, andeach signal accumulator outputs only one filter; a signal intensity ofan output end of the signal accumulator represents an accumulative valueof a signal intensity collected by the signal collector over a pastperiod of time, and when the signal intensity exceeds a specificthreshold, the filter outputs a pulse signal.

The pulse signal is represented by a binary 1; a pulse sequence of alocal spatial position corresponding to a signal collector is a binarysequence in which 1 appears intermittently, and a time interval betweentwo 1s in the sequence indicates required time for accumulating a later1 of the two 1s; and a binary sequence, in which all numbers are 1,indicates that a signal at the local spatial position corresponding tothe signal collector is always in a highest intensity state; and

all binary sequences are arranged into a binary sequence array accordingto spatial positions.

A reconstructed image at time to is a pixel value of I at position (i,j), which is a number of 1s appeared in a past Δ t of a correspondingbinary sequence, and Δ t is set as needed.

The method further comprises: representing a binary sequence in acompact way with fewer bits according to a statistical correlationbefore and after the binary sequence.

The method further comprises: recoding the binary sequence arrayaccording to a statistical correlation between adjacent and closesequences in space.

The method further comprises: recovering the binary sequence arrayaccording to an inverse process of a compact bit stream generationprocess.

According to another aspect of the disclosure, a device for encodingspace-time signals is provided, which includes a signal collector, atime-domain coding module and a space-domain coding module;

wherein the signal collector is used to collect a space-time signal ateach one of local spatial positions in a monitoring area, and the localspatial positions constitute the monitoring area;

the time-domain coding module is used to perform time-domain coding onthe space-time signal of the each local spatial position, so as toobtain a pulse sequence representing a change process of the space-timesignal at the each local spatial position; and

the space-domain coding module is used to perform space-domain coding onpulse sequences of all of the local spatial positions according to aspatial position relation to obtain a pulse sequence array.

The time-domain coding module comprises:

a signal accumulator, used to accumulate the space-time signal at theeach local spatial position according to time to obtain a cumulativesignal intensity value;

a filter, used to transform the cumulative signal intensity value, andoutput a pulse signal when a transformation result exceeds a specificthreshold; and

a processing unit, used to arrange pulse signals corresponding to theeach local spatial position into a sequence according to time, so as toobtain the pulse sequence representing the change process of thespace-time signal at the each local spatial location.

The signal collector is specifically used to collect the space-timesignal from a designated local spatial position to complete time-domainsampling; and multiple signal collectors are arranged into an array andcooperate with each other to cover the monitoring area, and completespace-domain sampling of the monitoring area.

The space-time signal is an optical signal, and the signal collector isa photosensitive device.

An electrical signal intensity output by the signal collector through aphotoelectric conversion is positively correlated to a collected lightintensity; the signal collector is connected to one or more signalaccumulators, and the signal collector transmits the electrical signalintensity to the connected signal accumulators;

the signal accumulator accumulates a signal in a past period of time,and an output end of the signal accumulator is the cumulative signalintensity value; one signal accumulator is connected to one filter, andthe signal accumulator transmits the cumulative signal intensity valueto the connected filter; and

the filter takes one or more signal accumulator as input, transformsinput cumulative signal intensity values according to a specific filterfunction, and outputs a pulse signal corresponding to a local spatialposition when a transformation result exceeds the specific threshold.

When the signal collector is connected to multiple signal accumulators,the signal collector outputs the electrical signal intensity to alldownstream signal accumulators at a same time; or, the electrical signalintensity is evenly assigned to all downstream signal accumulators; or,the electrical signal intensity is assigned to all downstream signalaccumulators according to a certain weight, and the weight is a functionof a spatial position distance between the multiple signal accumulatorsand the signal collector.

The multiple signal accumulator are time-limited rolling signalaccumulators or time-unlimited signal accumulator, the time-limitedrolling signal accumulators only accumulate a signal in a specificperiod of time before a current time, and an earlier signal isautomatically cleared; and the time-unlimited signal accumulatorsimplement accumulation continuously.

The filter function of the filter is set according to a sparsity of aspatial position that the filter needs to capture, and the sparsity ofthe spatial position that the filter needs to capture is determinedaccording to a local spatial scope of a signal collected by a signalcollector associated with the filter; and/or

the monitoring area is covered by using multiple filters through aredundant design, and a sparse mode of the each local spatial positionin the monitoring area can be captured by a corresponding filter; amulti-level redundancy design is adopted on space scale coverage for themultiple filters, the filters of different levels are sensitive tospatial sparsity of corresponding space scales, and an effective captureof any scale sparsity in the monitoring area is realized by acooperation of the multi-level filters; and/or

the filter transforms cumulative signal intensity values from the signalaccumulator according to a set filter function, so as to obtaintransformation coefficients related to signal distribution within alocal spatial position, when a transformation coefficient exceeds apreset threshold, the filter outputs a pulse signal which carries apulse intensity, and there is a corresponding relationship between thepulse intensity and the cumulative signal intensity value; and/or

after the filter outputs pulse signals, all signal accumulatorscorresponding to the filter are reset.

The filter is a binary filter, and the filter function is a thresholdfunction;

when there is only one signal accumulator as an input of the binaryfilter, if the cumulative signal intensity value input by the signalaccumulator exceeds a specified threshold, the binary filter outputs apulse signal, otherwise no pulse signal is not output;

when there are multiple signal accumulators as inputs of the binaryfilter, the binary filter implements a simple accumulation or a weightedaccumulation according to certain rules, and performs a filteringtransformation for cumulative signal intensity values input by themultiple signal accumulators, if a transformation result exceeds aspecified threshold, the binary filter outputs a pulse signal, otherwiseno pulse signal is output.

The device further comprises: a transformation coefficient processingmodule, configured to:

perform time-domain discrete representation for pulse outputs of thefilter by using a high frequency clock, so that the pulse outputs of thefilter occur only at times of equal intervals, transformationcoefficients output by all filters at a same time form a sparse array, atransformation coefficient corresponding to filters without output atthe time is 0, and sparse arrays are arranged into a transformationcoefficient array in a sequence of equal time intervals.

Pulses output binary filter are represented by binary numbers, when thebinary filter outputs a pulse, it is represented by 1, otherwise it isrepresented by 0; at a same time, a binary filter with a pulse outputoutputs 1, and a binary filter without pulse output outputs 0; and

the space-domain coding module is further configured to: outputs of allbinary filters at a same time constitute a binary sparse array accordingto a filter array, binary sparse arrays at all times are arranged into abinary sparse sequence array according to a sequence of equal timeintervals specified by a clock.

The signal collector, the signal accumulator and the filter constitute a“collection-accumulation-transformation” triplet in a one-to-one form,that is, each signal collector outputs only one signal accumulator, andeach signal accumulator outputs only one filter; a signal intensity ofan output end of the signal accumulator represents an accumulative valueof a signal intensity collected by the signal collector over a pastperiod of time, and when the signal intensity exceeds a specificthreshold, the filter outputs a pulse signal.

The pulse signal is represented by binary 1; a pulse sequence of a localspatial position corresponding to the signal collector is a binarysequence in which 1 appears intermittently, and a time interval betweentwo 1s in the sequence indicates required time for accumulating a later1 of the two 1s; and a binary sequence, in which all numbers are 1,indicates that a signal at the local spatial position corresponding tothe signal collector is always in a highest intensity state; and

all binary sequences are arranged into a binary sequence array accordingto spatial positions.

The signal collector is a photosensitive device, and all signalcollectors are arranged into a photosensitive array, the signalaccumulator is a photoelectric conversion circuit with a timeaccumulation function, the filter is a binary pulse filter, and thesignal accumulator and the filter constitute a time-delay binary pulsefilter, the device is a new type of camera device, in which an imagingunit works independently, and indicates a signal intensity correspondingto a local spatial position by outputting a pulse when a collected lightintensity reaches a threshold.

An operating frequency of the filter is higher than 1000 Hz.

The signal collector is a high-sensitivity photosensitive device, thesignal accumulator is a high-sensitivity converter, the signal collectorand the signal accumulator cooperate to accurately measure a quantity ofcollected photons, a time interval of pulses output from the filter isat a picosecond level, and an output binary sequence array represents anumber of photon irradiation in the monitoring area.

From the technical scheme provided by the embodiments of the presentdisclosure, it can be seen that the embodiments of the present are basedon the idea of performing time-domain encoding and then space-domainencoding to propose that a space-time signal of a local spatial positionis accumulated according to periods of time, a cumulative signalintensity value of the local spatial position is transformed accordingto a spatial sparsity, and a pulse signal corresponding to the localspatial position is output. Thus, a change process of each local spatialposition can be preserved, and a movement process of a high-speed movingobject can be finely reconstructed, which can provide an abundantinformation source for subsequent motion analysis and object detectionand tracking, while the frame rate is the upper limit of the changeinformation preserved in the traditional video. The image at any timecan be reconstructed, while the traditional video only retains the imageat the frame sampling time.

In another aspect, the present disclosure provides an imaging method,including: acquiring, at a specified imaging time, a pulse sequence in atime period before the specified imaging time, with regard to each pixelof a plurality of pixels; calculating a pixel value of the pixelaccording to the pulse sequence; obtaining an image at the specifiedimaging time according to a space arrangement of the pixels, inaccordance with pixel values of the plurality of pixels at the specifiedimaging time.

According to an embodiment of the present disclosure, the calculating apixel value of the pixel according to the pulse sequence includes:calculating a quantity of pulses in a fixed period of time before thespecified imaging time; obtaining the pixel value according to thequantity of pulses.

According to an embodiment of the present disclosure, the calculating apixel value of the pixel according to the pulse sequence includes:acquiring a length of time interval between adjacent pulses in the pulsesequence, calculating an intensity value corresponding to the timeinterval according to the length of the time interval, and obtaining afirst accumulated pixel value according to one or more intensity values.The obtaining an image at the specified imaging time according to aspace arrangement of the pixels, in accordance with pixel values of theplurality of pixels at the specified imaging time includes: obtainingthe image at the specified imaging time according to the firstaccumulated pixel value at a pixel position corresponding to each pixelof the plurality of pixels.

According to an embodiment of the present disclosure, the acquiring alength of time interval between adjacent pulses in the pulse sequence,calculating an intensity value corresponding to the time intervalaccording to the length of the time interval, and obtaining a firstaccumulated pixel value according to one or more intensity valuesincludes: accumulating at least one intensity value before the specifiedimaging time to obtain the first accumulating pixel value.

According to an embodiment of the present disclosure, the obtaining animage at the specified imaging time according to a space arrangement ofthe pixels, in accordance with pixel values of the plurality of pixelsat the specified imaging time, includes: setting a first specific amountcorresponding to the specified imaging time of the pixel position, andsumming the first specific amount and the first accumulated pixel valueto obtain a first pixel value of the pixel position; and obtaining theimage at the specified imaging time according to the first specificamount at pixel positions corresponding to the plurality of pixels.

According to an embodiment of the present disclosure, the obtaining animage at the specified imaging time according to a space arrangement ofthe pixels, in accordance with pixel values of the plurality of pixelsat the specified imaging time, includes: comparing the first pixel valuewith a pixel threshold range, and obtaining a second specific amount byadjusting the first specific amount; summing the first accumulated pixelvalue and the second specific amount to obtain a second pixel value ofthe pixel position; and obtaining the image at the specified imagingtime according to the second pixel value at pixel positionscorresponding to the plurality of pixels.

According to an embodiment of the present disclosure, the imaging methodfurther including: determining an accumulated time duration before thespecified imaging time. The acquiring a length of time interval betweenadjacent pulses in the pulse sequence, calculating an intensity valuecorresponding to the time interval according to the length of the timeinterval, and obtaining a first accumulated pixel value according to oneor more intensity values, includes: acquiring at least one intensityvalue corresponding to at least one time interval during the accumulatedtime duration; calculating at least one attenuation value of the atleast one intensity value after it is attenuated when reaching thespecified imaging time; summing the at least one attenuation value toobtain the first accumulated pixel value.

According to an embodiment of the present disclosure, the imaging methodfurther including: setting the first accumulated pixel value at 0 ifthere is no fired pulse in the accumulated time duration.

According to an embodiment of the present disclosure, an input pulsesignal at the pixel position can change first accumulated pixel valuesor first specific amounts at one or more other pixel positions.

According to an embodiment of the present disclosure, a first pixelvalue at the pixel position can change second specific amounts or pixelvalues at one or more other pixel positions.

According to an embodiment of the present disclosure, the comparing thefirst pixel value with a pixel threshold range, and obtaining a secondspecific amount by adjusting the first specific amount includes:obtaining the second specific amount by adjusting the value of the firstspecific amount if the first pixel value is not within the pixelthreshold range; and determining the first specific amount as the secondspecific amount if the first pixel value is within the pixel thresholdrange.

According to an embodiment of the present disclosure, the pixelthreshold range is set as a fixed value.

According to an embodiment of the present disclosure, the pixelthreshold range is determined based on normalized global pixel values.

According to an embodiment of the present disclosure, the pixelthreshold range is determined based on an ideal dynamic range of image.

According to an embodiment of the present disclosure, setting the firstspecific amount includes: setting the first specific amount as a fixedvalue.

According to an embodiment of the present disclosure, the at least oneintensity value is a function in which the at least one pixel value isattenuated at a fixed ratio with the time interval.

According to an embodiment of the present disclosure, the at least oneintensity value is o a function in which the at least one pixel value isattenuated at a fixed magnitude with the time interval.

According to an embodiment of the present disclosure, the at least onepixel value is a function in which the at least one pixel value isattenuated by being decreased with a decreasing part of the functionwith the time interval.

According to an embodiment of the present disclosure, the obtaining theimage at the single pulse-firing time according to the second pixelvalue at pixel positions corresponding to the plurality of pixelsincludes: filtering the second pixel value based on a temporal neighborrelationship of the pixel values at the pixel position; obtaining theimage at the specified imaging time according to the filtered secondpixel value.

According to an embodiment of the present disclosure, the obtaining theimage at the single pulse-firing time according to the second pixelvalue at pixel positions corresponding to the plurality of pixelsincludes: filtering the second pixel value based on a spatial neighborrelationship of the pixel values at the pixel position; obtaining theimage at the specified imaging time according to the filtered secondpixel value.

In another aspect, the present disclosure provides an imaging device,including: at least one processor; and at least one memory includingcomputer program code, the at least one memory and the computer programcode configured to, with the at least one processor, cause the apparatusto perform at least the following: acquiring, at a specified imagingtime, a pulse sequence in a time period before the specified imagingtime, with regard to each pixel of a plurality of pixels; calculating apixel value of the pixel according to the pulse sequence; obtaining animage at the specified imaging time according to a space arrangement ofthe pixels, in accordance with pixel values of the plurality of pixelsat the specified imaging time. The additional aspects and advantages ofthe present disclosure will be partly given in the followingdescription, which will be obvious from the following description or beunderstood through the practice of the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

In order to clearly illustrate technical solutions of embodiments of thepresent disclosure, the drawings used in the description of theembodiments will be briefly described below. Obviously, the drawings inthe following descriptions are just some embodiments of the presentdisclosure. For those of ordinary skill in the art, other drawings alsocan be obtained according to these drawings without any creative work.

FIG. 1 is a processing flowchart illustrating a method for encodingspace-time signals according to a first embodiment of the presentdisclosure.

FIG. 2 is a specific implementation diagram illustrating a device forencoding space-time signals according to a second embodiment of thepresent disclosure.

FIG. 3 is a schematic diagram showing steps of an imaging methodaccording to the present disclosure;

FIG. 4 is a schematic diagram showing pixels and a pulse sequence of animaging method according to the present disclosure;

FIG. 5 is a schematic diagram showing a pulse sequence of an imagingmethod according to the present disclosure;

FIG. 6 is a schematic diagram showing an output image of an imagingmethod according to the present disclosure; and

FIG. 7 is a schematic diagram showing an imaging device according to thepresent disclosure.

FIG. 8 is a schematic diagram showing an imaging device according to thepresent disclosure.

In the figures, a signal collector 21, a signal accumulator 22 and afilter 23 are shown.

DETAILED DESCRIPTION

In order to make the purpose, technical means and advantages of thepresent disclosure clear, the present disclosure will be furtherdescribed in detail in conjunction with the accompanying drawings.

Those skilled in the art may understand that, unless specificallystated, singular forms “a”, “one”, “said” and “the” used herein may alsoinclude plural forms. It should be further understood that the word“including” used in the specification of the present disclosure meansthe presence of the feature, integer, step, operation, element and/orcomponent, but it does not exclude the presence or addition of one ormore other features, integers, steps, operations, elements, componentsand/or their combination. It should be understood that when an elementis referred to being “connected” or “coupled” to another element, it canbe directly connected or coupled to other elements, or there can be anintermediate component. In addition, the “connected” or “coupled” usedhere may include a wireless connection or coupling. The term “and/or”used here includes any unit and all combinations of one or more relatedlisted items.

Those skilled in the art can understand that, unless otherwise defined,all terms (including technical and scientific terms) used herein havethe same meanings as generally understood by those of ordinary skill inthe art to which the present disclosure belongs. It should also beunderstood that, terms such as those defined in universal dictionariesshould be understood as having meanings consistent with those in thecontext of the prior art, and unless defined as here, would not beinterpreted in an idealized or overly formal sense.

In order to make embodiments of the present disclosure easy to beunderstood, several specific embodiments as examples will be furtherexplained below with reference to the accompanying drawings, and thespecific embodiments do not constitute a limitation for the embodimentsof the present disclosure.

The First Embodiment

In order to fundamentally solve the problem of high efficient encodingof space-time signals such as in video, the present disclosure proposesa new scheme from two aspects of video representation and codingmethods. Different from the method of representing a dynamic image as asequence of image sequence (the “image” here is also called “frame”)since the appearance of the film and television, the present disclosurefirst collects space-time signal change of each local spatial positionconstituting a monitoring area (for the image, it is a pixel changeprocess), and performs time-domain coding according to time orderrespectively. For the dynamic image, this time-sequence signal is calleda “pixel stream”, which is used to obtain a pulse sequence ofrepresenting a change process of the space-time signal at the localspatial position; then, a pulse sequence matrix composed oftime-sequence signals of each local spatial position is spatiallyencoded according to the spatial position relation (Spatial Sparsity) toobtain a pulse sequence array. It needs to be particularly noted thatalthough multiple pixel streams are still arranged into an arrayaccording to spatial relative position, each pixel stream does not haveequal interval sampling and “frame alignment” as in a traditional video,but retain change information and time-domain sparsity. The spatialencoding for the pixel stream array is not a simple image encoding, butan encoding for signal accumulation in the past period of time withinthe spatial range. Thus, this method can achieve high-efficiency spatialinformation encoding while retaining time-domain process informationwith high precision, and subverts the traditional method of space-timesignal coding.

This embodiment provides a process of a method for encoding space-timesignals. As shown in FIG. 1, the method includes the followingprocessing steps:

S110: collecting a light signal in a monitoring area by a signalcollector, and calculating a signal intensity value of the light signaland transmitting the signal intensity value to a signal accumulator.

Each signal collector collects a space-time signal from a designatedlocal spatial position, generates a pulse sequence, and completestime-domain sampling; a plurality of signal collectors are arranged intoan array and cooperate with each other to cover the entire monitoringarea, and complete space-domain sampling of the monitoring area.

The space-time signal is an optical signal. The signal collector is aphotosensitive device to realize photoelectric conversion, and theintensity of the electrical signal at the output end is positivelycorrelated to the collected light intensity. Each photosensitive deviceis responsible for a small square local area, and all devices arearranged into a neat square array by ranks. A honeycomb pattern(hexagonal segmentation), triangular segmentation, or other arrangementsmay also be used, and in this case, the central locations of the signalcollectors may not be in a straight line. Each signal collectorcorresponds to a specific local spatial position, so the signalcollector itself identifies the local spatial position of the outputoptical signal.

Each the signal collector collects the space-time signal of the localspatial position according to a set collection time interval. The framerate of the current common cameras is 24 to 120 (frames/second), thatis, the time interval is tens of milliseconds. The time interval of thepresent disclosure is obviously shorter, and it can be milliseconds,microseconds, nanoseconds and even picoseconds as required.

A signal collector is connected to one or more signal accumulators, andthe signal collector transmits the signal intensity value to the signalaccumulators to which it is connected.

S120: calculating a cumulative signal intensity value of each localspace position in a past period of time by the signal accumulator, andoutputting the cumulative signal intensity value to a filter.

The signal accumulator accumulates the signal in the past period oftime, and its output is the cumulative signal intensity value.

A signal collector can output a signal to one or more signalaccumulators.

A signal accumulator is connected to only one filter as an input of thefilter.

A filter can receive inputs from one or more signal accumulators. Thatis, a fan-out of the signal accumulator is 1, while a fan-in of thefilter can be 1 or greater.

The simplest case is that the signal collector, the signal accumulatorand the filter are in a one-to-one correspondence: one signal collectoracts as and only as an input of one signal accumulator, and the signalaccumulator is then connected to one filter and acts as and only as aninput of the filter. The filter only accepts the input of the signalaccumulator and does not accept inputs of other signal accumulators.

When a signal collector fans out a plurality of signal accumulators,there are at least three ways to transmit the signal intensity value:when the signal collector is connected to a plurality of signalaccumulators, the signal collector outputs the same signal intensityvalue to all downstream signal accumulators at the same time; or, thesignal intensity value is evenly assigned to all downstream signalaccumulators; or, the signal intensity value is assigned to alldownstream signal accumulators according to a certain weight, and theweight is a function of spatial position distances between the signalaccumulators and the signal collector.

The signal accumulator is a time-limited rolling signal accumulator or atime-unlimited signal accumulator. The time-limited rolling signalaccumulator only accumulates signal within a specific period of timebefore the current time, and the earlier signal is automaticallycleared; and the time-unlimited signal accumulator implementsaccumulation continuously.

The signal accumulator transfers cumulative signal intensity value to afilter to which it is connected. The filter takes one or more signalaccumulators as inputs and transforms the input cumulative signalintensity values according to a specific filter function.

S130: transforming the cumulative signal intensity values from thesignal accumulator according to a specific filter function by thefilter, so as to obtain transformation coefficients related to signaldistribution within the local spatial position. When a transformationcoefficient exceeds a set threshold, the filter outputs a pulse signalrepresented by a numerical value corresponding to the local spatialposition. The pulse signal carries pulse intensity, and there is acorresponding relationship between the pulse intensity and thecumulative signal intensity value. The filter obtains the local spatialposition information of the input signal according to the connectedsignal accumulator.

After the filter outputs the pulse signal, all the signal accumulatorsof the filter are reset.

In order to capture spatial sparse mode of any location and any scale asmuch as possible, the filter function of the filter is set according tosparsity of a spatial position that the filter needs to capture, and thesparsity of the spatial position that the filter needs to capture isdetermined according to a local spatial scope of a signal collected bythe signal collector associated with the filter. The monitoring area iscovered by using multiple filters through a redundant design, and asparse mode of each local spatial position in the monitoring area can becaptured by a corresponding filter. A multi-level redundancy design isadopted on space scale coverage for the multiple filters. The filters ofdifferent levels are sensitive to spatial sparsity of correspondingspace scales. An effective capture of any scale sparsity in themonitoring area is realized by a cooperation of the multi-level filters.

The simplest case is that a filter only accepts the input from a signalaccumulator. A filter may also receive inputs from a plurality of signalaccumulators. Each filter transforms cumulative signal intensity valuescorresponding to local spatial position from the respective signalaccumulator according to a specific filter function, and obtainstransformation coefficients corresponding to the local spatial position.

A simple form of filter is a binary filter, that is, the filter functionis a threshold function. When there is only a signal accumulator as theinput of the binary filter, if the cumulative signal intensity valueinput by the signal accumulator exceeds a specified threshold, thebinary filter outputs a pulse signal, otherwise no pulse signal isoutput.

When there are multiple signal accumulators as inputs of the binaryfilter, the binary filter implements a simple accumulation or a weightedaccumulation according to certain rules, and performs a filteringtransformation for cumulative signal intensity values input by themultiple signal accumulators. If the transformation result exceeds aspecified threshold, the binary filter outputs a pulse signal, otherwiseno pulse signal is output. In the weighted accumulation process of thecumulative signal intensity values, the earlier the collection time, thelower the signal weight.

The pulses output the binary filter are represented by binary numbers.When the filter outputs a pulse, it is represented by 1, otherwise it isrepresented by 0. At the same time, the binary filter with a pulseoutput outputs 1, and the binary filter without pulse output outputs 0.All these outputs at the same time constitute a binary sparse arrayaccording to the filter array. Binary coefficient arrays at all timesare arranged into a binary sparse sequence array according to a sequenceof equal time intervals specified by the clock, as an efficient binaryexpression of the space-time signal in the monitoring area.

The signal collector, the signal accumulator and the filter constitute a“collection-accumulation-transformation” triplet in a one-to-one form.That is, each signal collector outputs only one signal accumulator, andeach signal accumulator outputs only one filter. The signal intensity atan output end of the signal accumulator represents an accumulative valueof signal intensity collected by the signal collector over a past periodof time. When the intensity exceeds a specific threshold, the filteroutputs a pulse which is represented by a binary 1. In this way, thedynamic signal of the local spatial position corresponding to the signalcollector is transformed to a binary sequence in which 1 appearsintermittently, and a time interval between two 1s in the binarysequence indicates required time for accumulating a later 1 of the two1s; and a binary sequence, in which all numbers are 1, indicates thatthe signal at the local spatial position corresponding to the signalcollector is always in a highest intensity state.

Binary sequences generated by all“collection-accumulation-transformation” triplets are arranged into abinary sequence array according to the corresponding local spatialpositions, as an efficient binary expression of the space-time signalsin the monitoring area.

The signal collector is a photosensitive device, and all signalcollectors are arranged into a photosensitive array. The signalaccumulator is a photoelectric conversion circuit with a timeaccumulation function. The filter is a binary pulse filter, and thesignal accumulator and the filter constitute a time-delay binary pulsefilter. The device is a new type of camera device, in which an imagingunit (i.e., a “signal collector-signal accumulator-filter” group asmentioned above) works independently, and indicates a signal intensitycorresponding to a local spatial position by outputting a pulse(binary 1) when the collected light intensity reaches a threshold.

The filter function can be more complex, such as a LoG (Laplacian ofGaussian) filter, which is most sensitive to a speckle type input. Thefilter function of a filter bank may be a family of functions thatsatisfies a certain relationship, and a typical example is a family ofwavelet transform functions.

When a transformation coefficient corresponding to a certain localspatial position calculated by the filter exceeds a set threshold, thefilter outputs a pulse signal reflecting the signal intensity of thecorresponding local spatial position. The pulse signal can carry pulseintensity information, and the pulse intensity corresponds to thecumulative signal intensity value. When a transformation coefficientcorresponding to a certain local spatial position does not exceed a setthreshold, the filter does not output the pulse signal, and a low-levelsignal may be chose to output.

After the filter outputs the pulse signal, all signal accumulators ofthe filter are reset.

S140: arranging pulse signals corresponding to the local spatialposition into a sequence in time order to obtain a pulse sequenceexpressing the local spatial position signal and a change processthereof; and arranging the pulse sequences of all the local spatialpositions into a pulse sequence array according to the spatial positionrelation, which is served as an encoding for dynamic space-time signalsof the monitoring area.

Each filter in the filter array sets its own output pulse signalaccording to its own threshold, and outputs between filters may not besynchronized. In this way, an encoding with the time-domaincharacteristic for the cumulative signal intensity value correspondingto the local spatial position is realized.

In practical applications, the filter can be an analog filter, and itsoutput is a pulse sequence array.

In practical applications, the pulse signal output from the filter mayonly carry one bit of information, namely 0 (no pulse output) or 1 (withpulse output). A pulse sequence matrix is degraded to a bit sequencearray, which is called a bit stream array.

In practical applications, an output of the signal accumulator is avalue in a certain range. The filter is a digital filter and outputs avalue in a certain range. The value output from the filter may have onlytwo states: 0 (no output) or 1 (with output).

The pulse output of the filter is time-domain discretely expressed byusing a high frequency clock, which makes the pulse outputs of thefilter occur only at the time of equal intervals. Transformationcoefficients output by all filters at the same time form a sparse array,and transformation coefficients corresponding to the filters withoutoutput at this time are 0. Sparse arrays are arranged into atransformation coefficient array in a sequence of equal time intervalsas an efficient expression of the space-time signals in the monitoringarea.

An operating frequency of the filter is higher than 1000 Hz, that is, adiscrete time interval at which the filter is allowed to output is lessthan 1 millisecond, and may reach a microsecond, nanosecond or evenpicosecond level.

The signal collector is a high-sensitivity photosensitive device, andthe signal accumulator is a high-sensitivity converter. Cooperating ofthe signal collector and the signal accumulator may accurately measurethe quantity of collected photons. The time interval of pulses outputfrom the filter is at a picosecond level, and the output binary sequencearray represents the number of photon irradiation in the monitoringarea.

At any time, a transformation coefficient array in the past period oftime is inversely transformed by an inverse converter groupcorresponding to the filter, and space-time signals in the current timeare reconstructed.

On the basis of a coefficient array arranged at the previous time, theexisting transformation coefficient at the same position is replaced bythe transformation coefficient at the current time, and the spatialsignal at the current time is reconstructed. The dynamic signals arereconstructed by analogy.

The reconstructed image of time to is a pixel value of I at position (i,j), which is the number of 1 occurred in the corresponding binarysequence in the past Δ t, and Δ t may be set arbitrarily as needed.

The binary sequence is compactly represented by using fewer bitsaccording to a statistical correlation before and after the binarysequence, which includes, but is not limited to, run-length encoding orarithmetic encoding, etc.

The binary sequence array is recoded according to a statisticalcorrelation between adjacent and close sequences in space, such asarithmetic encoding, so as to reduce the number of bits used.

The binary sequence array is restored according to an inverse process ofa compact bit stream generation process.

According to the regularity of the above-mentioned transformationcoefficient array or the pulse sequence matrix, prediction coding,run-length coding, entropy coding and other methods are used forcompression to form a more efficient compressed bit stream. An originalcoefficient sequence array is obtained by using a corresponding decodingalgorithm for the compressed bit stream.

By analyzing the time-sequence characteristics of the pulse sequencearray, the object motion information contained in the input signals maybe obtained, and the description of the position and movement process ofthe object may be obtained. The analysis for the coefficient sequencearray may detect an object and attribute the object included in thespace-time signals.

The Second Embodiment

This embodiment provides a device for encoding space-time signals. Aspecific implementation structure of the device includes a signalcollector 21, a time-domain coding module and a space-domain codingmodule.

The signal collector 21 is used for collecting a space-time signal ofeach of local spatial positions in a monitoring area, wherein the localspatial positions constitute the monitoring area.

The time-domain coding module is used to perform time-domain coding onthe space-time signal of the each local spatial position, so as toobtain a pulse sequence representing a change process of the space-timesignal at the each local spatial position.

The space-domain coding module is used to perform space-domain coding onpulse sequences of all local spatial positions according to a spatialposition relation to obtain a pulse sequence array.

In an embodiment of the present disclosure, as shown in FIG. 2, thetime-domain coding module includes: a signal accumulator 22, a filter 23and a processing unit.

The signal accumulator 22 is used to accumulate the space-time signal atthe local spatial position according to the time to obtain a cumulativesignal intensity value;

the filter 23 is used to transform the cumulative signal intensityvalue, and output a pulse signal when a transformation result exceeds aspecific threshold; and

the processing unit is used to arrange the pulse signals correspondingto the local spatial position into a sequence in time order to obtain apulse sequence representing a change process of the space-time signal atthe local spatial location.

In an embodiment of the present disclosure, the signal collector 21 isspecifically used to collect the space-time signal from a designatedlocal spatial position to complete time-domain sampling. A plurality ofsignal collectors 21 are arranged into an array to cooperate with eachother to cover the monitoring area, and complete space-domain samplingof the monitoring area.

In an embodiment of the present disclosure, the space-time signal is anoptical signal, and the signal collector 21 is a photosensitive device.

An electrical signal intensity, output by the signal collector 21through photoelectric conversion, is positively related to the collectedlight intensity. One signal collector 21 is connected to one or moresignal accumulators 22 transfers the electrical signal intensity to theconnected signal accumulators 22.

The signal accumulator 22 accumulates a signal in the past period oftime, and an output end of the signal accumulator 22 is the cumulativesignal intensity value. A signal accumulator 22 is connected to a filter23 and transfers the cumulative signal intensity value to the connectedfilter 23.

The filter 23 takes one or more signal accumulators 22 as input, andtransforms the input cumulative signal intensity values according to aspecific filter function. When a transformation result exceeds aspecific threshold, the filter 23 outputs a pulse signal correspondingto the local spatial position.

In an embodiment of the present disclosure, when the signal collector 21is connected to multiple signal accumulators 22, the signal collector 21outputs a same electrical signal intensity to all downstream signalaccumulators 22 at the same time; or, the electrical signal intensity isevenly assigned to all downstream signal accumulators 22; or, theelectrical signal intensity is assigned to all downstream signalaccumulators 22 according to a certain weight, and the weight is afunction of spatial position distances between the signal accumulators22 and the signal collector 21.

In an embodiment of the present disclosure, the signal accumulator 22 isa time-limited rolling signal accumulator 22 or a time-unlimited signalaccumulator 22. The time-limited rolling signal accumulator 22 onlyaccumulates signal in a specific period of time before the current time,and the earlier signal is automatically cleared; and the time-unlimitedsignal accumulator 22 implements accumulation continuously.

In an embodiment of the present disclosure, the filter function of thefilter 23 is set according to sparsity of a spatial position that thefilter 23 needs to capture, and the sparsity of the spatial positionthat the filter 23 needs to capture is determined according to a localspatial scope of a signal collected by the signal collector 21associated with the filter 23; and/or

the monitoring area is covered by using multiple filters 23 through aredundant design, and a sparse mode of each local spatial position inthe monitoring area can be captured by a corresponding filter 23;and/or,

a multi-level redundancy design is adopted on space scale coverage forthe multiple filters 23, filters 23 of different levels are sensitive tospatial sparsity of corresponding space scales, and an effective captureof any scale sparsity in the monitoring area is realized by acooperation of the multi-level filters 23; and/or,

the filter 23 transforms the cumulative signal intensity value from thesignal accumulator 22 according to a set filter function to obtaintransformation coefficients related to signal distribution within thelocal spatial position; when a transformation coefficient exceeds apreset threshold, the filter 23 outputs a pulse signal, the pulse signalcarries a pulse intensity, and there is a corresponding relationshipbetween the pulse intensity and the cumulative signal intensity value;and/or,

after the filter 23 outputs the pulse signal, all signal accumulators 22corresponding to the filter 23 are reset.

In an embodiment of the present disclosure, the filter 23 is a binaryfilter 23, and the filter function is a threshold function.

When there is only one signal accumulator 22 as an input of the binaryfilter 23, if the cumulative signal intensity value input by the signalaccumulator 22 exceeds a specified threshold, the binary filter 23outputs a pulse signal, otherwise no pulse signal is output.

When there are a plurality of signal accumulators 22 as inputs of thebinary filter 23, the binary filter 23 implements a simple accumulationor a weighted accumulation according to certain rules, and performs afiltering transformation for cumulative signal intensity values input bythe plurality of signal accumulators 22. If the transformation resultexceeds a specified threshold, the binary filter 23 outputs a pulsesignal, otherwise no pulse signal is output.

In an embodiment of the present disclosure, the device further includes:a transformation coefficient processing module, which is configured to:

discretely represent pulse outputs of the filter 23 in time domain by ahigh frequency clock, so that the pulse outputs of the filter 23 occuronly at equal intervals, transformation coefficients output by allfilters 23 at the same time form a sparse array, the transformationcoefficients corresponding to filters 23 without output at this time are0, and sparse arrays are arranged at equal time intervals into atransformation coefficient array as an expression for the space-timesignals of the monitoring area.

In an embodiment of the present disclosure, the pulse output the binaryfilter 23 is represented by a binary number. When the binary filter 23outputs a pulse, it is represented by 1, otherwise it is represented by0; at the same time, the binary filter 23 with a pulse output outputs 1,and the binary filter 23 without output outputs 0.

The space-domain coding module is further configured to: outputs of allbinary filters 23 at the same time constitute a binary sparse arrayaccording to an array of filters 23; binary sparse arrays at all timesare arranged into a binary sparse sequence array according to a sequenceof equal time intervals specified by a clock, which acts as a binaryexpression of the space-time signals in the monitoring area.

In an embodiment of the present disclosure, the signal collector 21, thesignal accumulator 22 and the filter 23 constitute a“collection-accumulation-transformation” triplet in a one-to-one form.That is, each signal collector 21 outputs only one signal accumulator22, and each signal accumulator 22 outputs only one filter 23. Thesignal intensity at an output end of the signal accumulator 22represents an accumulative value of signal intensity collected by thesignal collector 21 over a past period of time. When the signalintensity exceeds a specific threshold, the filter 23 outputs a pulsesignal.

In an embodiment of the present disclosure, the pulse signal isrepresented by a binary 1. The pulse sequence of the local spatialposition corresponding to the signal collector 21 is a binary sequencein which 1 appears intermittently, and a time interval between two 1 sin the sequence indicates required time for accumulating a later 1 ofthe two 1s. A binary sequence, in which all the numbers are 1, indicatesthat the signal at the local spatial position corresponding to thesignal collector 21 is always in a highest intensity state.

All binary sequences are arranged into a binary sequence array accordingto the spatial positions as a binary expression of the space-timesignals in the monitoring area.

In an embodiment of the present disclosure, the signal collector 21 is aphotosensitive device, and all signal collectors 21 are arranged into aphotosensitive array. The signal accumulator 22 is a photoelectricconversion circuit with a time accumulation function. The filter 23 is abinary pulse filter 23, and the signal accumulator 22 and the filter 23constitute a time-delay binary pulse filter 23. The device is a new typeof camera device, in which an imaging unit works independently, andoutputs a pulse to indicate a signal intensity corresponding to a localspatial position when the collected light intensity reaches a threshold.An operating frequency of the filter 23 is higher than 1000 Hz.

In an embodiment of the present disclosure, the signal collector 21 is ahigh-sensitivity photosensitive device, and the signal accumulator 22 isa high-sensitivity converter. The signal collector 21 and the signalaccumulator 22 cooperate to accurately measure the quantity of collectedphotons. The time interval of pulses output from the filter 23 is at apicosecond level, and the output binary sequence array represents thenumber of photon irradiation in the monitoring area.

The specific process of encoding the space-time signals by the deviceaccording to the embodiments of the present disclosure is similar to theembodiments of the method described previously, and will not bedescribed in detail here.

In summary, the embodiments of the present are based on the idea ofperforming time-domain encoding and then space-domain encoding topropose that a space-time signal of a local spatial position isaccumulated according to periods of time, a cumulative signal intensityvalue of the local spatial position is transformed according to aspatial sparsity, and a pulse signal corresponding to the local spatialposition is output. Further, a sequence signal of the local spatialposition is obtained, and sequence signals of all local spatialpositions are arranged into a pulse sequence matrix. Thus, a codingmethod for space-time signals taking account of temporal information andspatial information simultaneously is provided.

The beneficial effects of the present disclosure include at least thefollowing:

1) a change process of each local spatial position can be preserved, anda movement process of a high-speed moving object can be finelyreconstructed, which can provide an abundant information source forsubsequent motion analysis and object detection and tracking, while theframe rate is the upper limit of the change information preserved in thetraditional video;

2) an image at any time can be reconstructed: a static image at aspecific time is an accumulation of the change process over a pastperiod of time, with the present disclosure, the image at any time canbe reconstructed, while the traditional video only retains the image atthe frame sampling time;

3) a high dynamic image at any time and in any space window can bereconstructed: the recording of the traditional video is an accumulationof light changes between two frames, and the corresponding dynamic rangeis often limited and fixed; with the present disclosure, the light inany period of time and any space window range can be accumulated, andthe obtained dynamic range is determined by the light condition in theperiod of time and the space range, which is dynamic, and may be highdynamic;

4) it is beneficial for the design of time-domain compression algorithm:inter-frame prediction in traditional video compression involves complexmotion estimation and motion compensation calculation, while in thepresent disclosure, the time-domain information is directly implied inthe original code stream, which does not need to deliberately designcomplex algorithms such as the inter-frame prediction and the data suchas the encoding motion vector, moreover, since the code stream of thepresent disclosure is “continuous” in the time domain (the time intervalis also particularly small in the discrete mode, such as milliseconds oreven smaller), and the correlation is stronger, it is easier to designefficient encoding algorithms;

5) it is beneficial for improving the efficiency of the space-domaincompression: in the traditional video, the light changes in a period oftime (between two frames) are “squeezed” forcibly in an image, whichimproves the complexity of the image, and improves the difficulty of thespace-domain coding (mainly referring to the transform coding) in thetraditional video compression, the cost of the residual expression ismuch; in the present disclosure, the dynamic image is represented by thesequence array, which may continue to use the transform coding method tocompress (corresponding to the filter function and the family offunctions of the filter bank), filters are not “forced” to performtransform coding synchronously at the same time as traditional methods,but each filter decides whether to output at any time according to itsown input mode, so signal modes in the space domain may be capturedbetter, and the efficiency of the space-domain compression is improved.

The pulse signals involved in the present disclosure have the followingfeatures: collecting spatiotemporal signals of individual local spatialpositions in a monitored region, and accumulating the spatiotemporalsignals of the local spatial positions based on time to obtain aaccumulative signal intensity value; transforming the accumulativesignal intensity value by a filter, and outputting a pulse signal whenthe transformation result exceeds a certain threshold; arranging thepulse signals corresponding to the local spatial positions in a sequencein a chronological order to obtain a pulse sequence representing thelocal spatial position signal and its change; and arranging the pulsesequences of all the local spatial positions based on a spatial positionrelationship to obtain pulse array signals.

An imaging method is provided according to an embodiment of the presentdisclosure. As shown in FIG. 3, the imaging method includes: S310:acquiring, at a specified imaging time, a pulse sequence in a timeperiod before the specified imaging time, with regard to each pixel of aplurality of pixels; S320: calculating a pixel value of the pixelaccording to the pulse sequence; S330: obtaining an image at thespecified imaging time according to a space arrangement of the pixels,in accordance with pixel values of the plurality of pixels at thespecified imaging time.

In this embodiment, imaging may be directed to the forming of an imagewhich may be a color image or a black and white image. The plurality ofpixels may be directed to a plurality of pixel dots constituting aportion or the whole of an image. Each pixel of a plurality of pixelsmay be directed to each pixel dots of the plurality of pixel dotsconstituting a portion or the whole of an image. The specified imagingtime may be directed to a particular time during a continuous shootingprocess of shooting a subject; the appearance and lighting circumstanceof the shooting subject is specified at this time, and thus the imageformed by an imaging device is specified. In some embodiments, thespecified imaging time may be interpreted as the single pulse-firingtime as mentioned below. The pulse sequence may be directed to asequence formed along time by a series of pulses formed with regard toeach pixel. The pulse sequence may be formed by collectingspatiotemporal signals of individual local spatial positions in amonitored region, and accumulating the spatiotemporal signals of thelocal spatial positions based on time to obtain a accumulative signalintensity value; transforming the accumulative signal intensity value bya filter, and outputting a pulse signal when the transformation resultexceeds a certain threshold; arranging the pulse signals correspondingto the local spatial positions in a sequence in a chronological order toobtain a pulse sequence representing the local spatial position signaland its change.

In this embodiment, acquiring a pulse sequence in a time period beforethe specified imaging time, with regard to each pixel of a plurality ofpixels, may be directed to, when each pulse sequence corresponds to onepixel, acquiring a sequence of one or more pulses arranged along time,the one or more pulses being formed in a time period of a particularlength before the specified imaging time. In this embodiment,calculating a pixel value of the pixel according to the pulse sequencemay be directed to calculating a pixel value of a pixel according topulse distribution in a pulse sequence, such as the gap between adjacentpulses, the number of pulses in a fixed time interval, and the like. Inthis embodiment, obtaining an image at the specified imaging timeaccording to a space arrangement of the pixels, in accordance with pixelvalues of the plurality of pixels at the specified imaging time, may bedirected to performing the same calculating and processing with regardto each of the pixels to obtain a pixel value shown by each pixel of theplurality of pixels of the imaging device at the specified imaging time,and display the corresponding pixel value at respective positionaccording to the position of each pixel in relation to all the pixels ofthe imaging device, so as to constitute the whole image.

In an embodiment, calculating a pixel value of the pixel according tothe pulse sequence includes: calculating a quantity of pulses in a fixedperiod of time before the specified imaging time; obtaining the pixelvalue according to the quantity of pulses.

In this embodiment, calculating a quantity of pulses in a fixed periodof time before the specified imaging time may be directed to analyzingthe pulse sequence, finding the specific imaging time in the timeduration represented by the pulse sequence, extracting a fixed segmentof time period before the specific imaging time, and counting the numberof pulses in the fixed segment of time period. In this embodiment,obtaining the pixel value according to the quantity of pulses may bedirected to taking the number of pulses as a parameter, performing aparticular calculating to obtain the pixel value.

In an embodiment, calculating a pixel value of the pixel according tothe pulse sequence includes: acquiring a length of time interval betweenadjacent pulses in the pulse sequence, calculating an intensity valuecorresponding to the time interval according to the length of the timeinterval, and obtaining a first accumulated pixel value according to oneor more intensity values. Obtaining an image at the specified imagingtime according to a space arrangement of the pixels, in accordance withpixel values of the plurality of pixels at the specified imaging timeincludes: obtaining the image at the specified imaging time according tothe first accumulated pixel value at a pixel position corresponding toeach pixel of the plurality of pixels.

In this embodiment, acquiring a length of time interval between adjacentpulses in the pulse sequence may be directed to analyzing the pulsesequence, finding one or more pairs of adjacent pulses (no other pulseexists between adjacent pulses), and computing the length of timerepresented by the time interval between adjacent pulses. The intensityvalue may be directed to a value for representing the concentrationdegree of pulses. In some embodiment, the intensity value may beinterpreted to be a pixel value as an intermediate result of calculatingor processing, which is not necessarily the pixel value as finallyshown. In this embodiment, calculating an intensity value correspondingto the time interval according to the length of the time interval may bedirected to taking the length of the time interval as a parameter,substituting the parameter in a corresponding formulation to figure outthe intensity value. In this embodiment, obtaining a first accumulatedpixel value according to one or more intensity values may be directed toobtaining one or more intensity values with regard to one or more pairsof adjacent pulses; and performing a calculation based on the one ormore intensity values, such as accumulating these intensity values, orsubstituting these intensity values as parameters in a formulation, toobtain the first accumulated pixel value. In this embodiment, obtainingthe image at the specified imaging time according to the firstaccumulated pixel value at a pixel position corresponding to each pixelof the plurality of pixels may be directed to performing calculationwith regard to each pixel of the imaging device to obtain acorresponding first accumulated pixel value; and displaying according tothe result of calculation with regard to all the pixels to obtain theimage.

In an embodiment, acquiring a length of time interval between adjacentpulses in the pulse sequence, calculating an intensity valuecorresponding to the time interval according to the length of the timeinterval, and obtaining a first accumulated pixel value according to oneor more intensity values includes: accumulating at least one intensityvalue before the specified imaging time to obtain the first accumulatingpixel value.

In this embodiment, accumulating at least one intensity value before thespecified imaging time to obtain the first accumulating pixel value maybe directed to performing calculation with regard to one or more pairsof adjacent pulses before the specific imaging time to obtain at leastone intensity value; and accumulating the at least one intensity valueto obtain the first accumulated pixel value according to theaccumulating result.

In an embodiment, obtaining an image at the specified imaging timeaccording to a space arrangement of the pixels, in accordance with pixelvalues of the plurality of pixels at the specified imaging timeincludes: setting a first specific amount corresponding to the specifiedimaging time of the pixel position, and summing the first specificamount and the first accumulated pixel value to obtain a first pixelvalue of the pixel position; and obtaining the image at the specifiedimaging time according to the first specific amount at pixel positionscorresponding to the plurality of pixels.

In an embodiment, obtaining an image at the specified imaging timeaccording to a space arrangement of the pixels, in accordance with pixelvalues of the plurality of pixels at the specified imaging timeincludes: comparing the first pixel value with a pixel threshold range,and obtaining a second specific amount by adjusting the first specificamount; summing the first accumulated pixel value and the secondspecific amount to obtain a second pixel value of the pixel position;and obtaining the image at the specified imaging time according to thesecond pixel value at pixel positions corresponding to the plurality ofpixels.

In an embodiment, the imaging method further includes: determining anaccumulated time duration before the specified imaging time. Acquiring alength of time interval between adjacent pulses in the pulse sequence,calculating an intensity value corresponding to the time intervalaccording to the length of the time interval, and obtaining a firstaccumulated pixel value according to one or more intensity values,includes: acquiring at least one intensity value corresponding to atleast one time interval during the accumulated time duration;calculating at least one attenuation value of the at least one intensityvalue after it is attenuated when reaching the specified imaging time;summing the at least one attenuation value to obtain the firstaccumulated pixel value.

In this embodiment, determining an accumulated time duration before thespecified imaging time may be directed to analyzing the pulse sequence,finding the specific imaging time from the time duration represented bythe pulse sequence, and extracting a fixed segment of time period beforethe specified imaging time as the accumulated time duration. In thisembodiment, acquiring at least one intensity value corresponding to atleast one time interval during the accumulated time duration may bedirected to finding at least one pair of adjacent pulses in theaccumulated time duration from the pulse sequence, and obtaining atleast one intensity value according to the time interval between the twopulses of the at least one pair of adjacent pulses. In this embodiment,calculating at least one attenuation value of the at least one intensityvalue after it is attenuated when reaching the specified imaging timemay be directed to calculating the intensity value according theadjacent pulses at early times, and calculating the attenuation value ofthe intensity value according to the length of time gap between theadjacent pulses and the specified imaging time. In an embodiment, theimaging method further includes: setting the first accumulated pixelvalue at 0 if there is no fired pulse in the accumulated time duration.

In an embodiment, an input pulse signal at the pixel position can changefirst accumulated pixel values or first specific amounts at one or moreother pixel positions.

In an embodiment, a first pixel value at the pixel position can changesecond specific amounts or pixel values at one or more other pixelpositions.

In an embodiment, a pulse signal-based display method is provided. Themethod includes:

analyzing a pulse sequence corresponding to a single pixel position toobtain pulse-firing information of the pulse sequence, the pulse-firinginformation including multiple pulse-firing times;

acquiring pixel values corresponding to multiple pulse-firing timesbefore a single pulse-firing time, and accumulating the pixel values asa first accumulated pixel value;

setting a first specific amount corresponding to the single pulse-firingtime of the pixel position, and summing the first specific amount andthe first accumulated pixel value to obtain a first pixel value of thepixel position;

comparing the first pixel value with a pixel threshold range, andobtaining a second specific amount by adjusting the first specificamount; and

obtaining a second pixel value of the pixel position by summing thefirst accumulated pixel value and the second specific amount.

The acquiring a first accumulated pixel value of pixel valuescorresponding to multiple pulse-firing times before a singlepulse-firing time includes:

determining an accumulated time duration before the single pulse-firingtime;

acquiring respective pixel values corresponding to the pulse-firingtimes during the accumulated time duration;

calculating respective attenuation values of the pixel values after theyare attenuated when reaching the single pulse-firing time; and

obtaining the first accumulated pixel value by summing the attenuationvalues.

The acquiring respective pixel values corresponding to the pulse-firingtimes during the accumulated time duration includes:

setting the first accumulated pixel value at 0 if there is nopulse-firing time in the accumulated time duration.

For a pixel at each position, if a pulse occurs at a current timeinstant t0, the pixel value I(t0) of the pixel at the current timeinstant changes as follows:

${I( t_{0} )} = {{I( t_{0} )} + {\frac{\Delta}{\tau}{\exp( {1 - \frac{\Delta}{\tau}} )}}}$

where I(t0) is an initial value (first pixel value) of the pixel, Δ is adelay factor, τ is a model parameter, and exp is an exponential functionwith a natural constant e being the base.

In a case where the pixel value is superimposed at the time instant whenthe pulse occurs, a superimposition amount is gradually attenuated overtime, and the pixel value at a time instant t (t>t0) is:

${I(t)} = {{I( t_{0} )} + {\frac{t - t_{0} - \Delta}{\tau}{\exp( {1 - \frac{t - t_{0} - \Delta}{\tau}} )}}}$

where I(t) is a value of the first pixel value after being attenuated atthe time instant t, and the upper and lower limits of the pixel valueand the attenuating rate can be adjusted by adjusting the modelparameter T and the attenuating factor T.

The comparing the first pixel value with a pixel threshold range, andobtaining a second specific amount by adjusting the first specificamount, includes:

The adjusting the value of the first specific amount includes:increasing or decreasing the value of the first specific amount, to makea value obtained by superimposing the value of the first specific amountwith the first accumulated pixel value be within the pixel thresholdrange. By adjusting the value of the first specific amount, theresulting second specific amount can express more information ascompared to the values of second specific amounts of all spatial pixels.

The pixel threshold range includes: a fixed value set as the pixelthreshold range, and/or a pixel threshold range determined based onnormalized global pixel values, and/or a pixel threshold rangedetermined based on an ideal dynamic range of image. Each pixel has arelatively independent pixel threshold range.

The determining the pixel threshold range based on normalized globalpixel values includes: based on pulse firing conditions of pixelpositions at the same time instant and based on first pixels, judgingwhether most of the first pixel values at the time instant is too smallor too large; if yes, normalizing the first pixel values based on theglobal pixel value to determine a pixel threshold range. In this way,the generated image is prevented from being too bright or too dark,which will cause unclear displayed content.

If the first pixel value is out of the pixel threshold range but is veryclose to the pixel threshold range, the pixel threshold range is changedto include the first pixel value.

The setting the first specific amount includes setting the firstspecific amount as a fixed value. The fixed value includes a functionwhich is attenuated over time.

The first specific amount is 0 when there is no pulse to fire.

The first specific amount can also be dynamically adjusted based on theaccumulated time duration before the single pulse-firing time. In a casewhere multiple dense pulse firings occur in the accumulated timeduration before the single pulse-firing time, the value of the firstspecific amount is dynamically reduced.

The pixel values are obtained based on a function in which the pixelvalues are attenuated with the pulse-firing time, including: attenuatedat a fixed ratio, attenuated at a fixed magnitude, and/or attenuated bybeing decreased with a decreasing part of the function.

The function includes the Gaussian function, the Exponential functionand the Logarithmic function, or includes a biological neuron model.

After the obtaining the second pixel value of the pixel position bysumming the first accumulated pixel value and the adjusted firstspecific amount, the method further includes: filtering the second pixelvalue based on a temporal neighbor relationship of the pixel values atthe single pixel position.

The method includes generating an image by using firstaccumulated/first/second pixel values of all pixel positions at the samepulse-firing time.

The generating an image by using first accumulated/first/second pixelvalues of all pixel positions at the same pulse-firing time includes:

directly restoring an image by using the first accumulated/first/secondpixel values of all pixel positions at the same pulse-firing time,and/or forming an image after the first accumulated/first/second pixelvalues are filtered, based on a spatial neighbor relationship of allpixel positions, and/or forming an image after the firstaccumulated/first/second pixel values are filtered, based on aspatiotemporal neighbor relationship of all pixel positions.

For generating an image by using the pixel values, a good effect isachieved when the image is generated by using the second pixel values.

Displaying a pulse signal of a certain pixel position at a certain timeinstant in the pulse data is taken as an example, and it is assumed thatthe certain time instant is t, and the certain pixel position is theposition of a pixel P.

A pulse sequence corresponding to the position of the pixel P isanalyzed to obtain pulse-firing information of the pulse sequence, andthe pulse-firing information includes multiple pulse-firing times. FIG.4 is a pulse sequence corresponding to the position of the pixel P.

A first accumulated pixel value of pixel values corresponding tomultiple pulse-firing times before the time instant t is acquired.

FIG. 5 shows the pulse sequence of the position P.

A historical time before the time instant t is determined based on thepulse-firing information. It is assumed that Δt is an accumulated timeduration before the time instant t.

Pixel values corresponding to pulse-firing times in the accumulated timeduration (Δt) are acquired. Attenuation values of the pixel values afterthey are attenuated when reaching the single pulse-firing time (the timeinstant t) are calculated.

The attenuation values are summed to obtain a first accumulated pixelvalue. As shown in FIG. 5, there are totally 14 accumulated timeinstants in the time duration Δt before the time instant t, which areset as time instants t1 to t14 respectively. Attenuation values of thepixel values after they are attenuated when reaching the singlepulse-firing time are calculated. There are pulses firing at the timeinstant t3, time instant t8 and time instant t10 respectively. It isassumed that the pulse at the time instant t3 is an accumulated firstpulse, the pulse at the time instant t8 is an accumulated second pulseand the pulse at the time instant t10 is an accumulated third pulse.

A specific amount is added at the time instant t3, time instant t8 andtime instant t10 respectively. A value of each specific amount at eachof the above time instants after it is attenuated when reaching the timeinstant t is calculated, and pixel values of the three accumulated timeinstants after attenuation are obtained. The three pixel values afterattenuation are summed to obtain a first accumulated pixel value.

The first accumulated pixel value is 0 if there is no pulse fired duringthe time duration Δ t.

A first specific amount corresponding to the single pulse-firing time(the time instant t) of the position P of the pixel is set, and thefirst specific amount and the first accumulated pixel value are summedto obtain a first pixel value of the position P of the pixel.

The first pixel value is compared with the pixel threshold range, andthe first specific amount is adjusted. The pixel threshold range of thepixel P is calculated by setting the pixel threshold range and/ordetermining the pixel threshold range based on normalized global pixelvalues and/or determining the pixel threshold range based on an idealdynamic range of image. Based on the pixel threshold range correspondingto the position P of the pixel, whether the first pixel value is withinthe range is judged. The value of the first specific amount is adjusted,and a second specific amount is obtained.

The first accumulated pixel value of the position P of the pixel and thesecond specific amount are summed to obtain a second pixel value of theposition P of the pixel.

The second pixel value of the position P of the pixel may be filteredbased on a temporal neighbor relationship as required. That is, secondpixel values of respective accumulated time instants of the position Pof the pixel before the time instant t are calculated based on atemporal neighbor relationship of pixel values at the position P of thepixel. Then, a curve of the second pixel values corresponding to therespective accumulated time instants is obtained. The second pixelvalues at the time instant t are filtered based on the curve to obtainthe filtered second pixel values.

The filtering based on the temporal neighbor relationship includes:filtering second pixel values of all global pixel positions based on thetemporal neighbor relationship, or only filtering second pixel values ofsome pixel positions based on the temporal neighbor relationship.

Second pixel values of all pixel positions at the time instant t arecalculated using the above method, and an image at the time instant t isformed using the second pixel values of all the pixel positions.

The forming an image at the time instant t using the second pixel valuesof all the pixel positions includes:

directly restoring an image by using the second pixel values of allpixel positions at the same pulse-firing time, and/or forming an imageafter the second pixel values are filtered, based on a spatial neighborrelationship of all pixel positions, and/or forming an image after thesecond pixel values are filtered, based on a spatiotemporal neighborrelationship of all pixel positions.

FIG. 6 is a clear image on a rotary disc rotating at high speed, whichis restored (generated) using pulse data.

An imaging device is further provided according to an embodiment of thepresent disclosure. As shown in FIG. 7, the device includes: anacquiring module 710, a calculating module 720 and a displaying module730 connected in sequence.

The acquiring module 710 is configured to acquire, at a specifiedimaging time, a pulse sequence in a time period before the specifiedimaging time, with regard to each pixel of a plurality of pixels.

The calculating module 720 is configured to calculate a pixel value ofthe pixel according to the pulse sequence.

The displaying module 730 is configured to obtain an image at thespecified imaging time according to a space arrangement of the pixels,in accordance with pixel values of the plurality of pixels at thespecified imaging time.

In an embodiment, an imaging device 800 is further provided. As shown inFIG. 8, the device 800 including: at least one processor 810; and atleast one memory 820 including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the device to perform at least the following:

acquiring, at a specified imaging time, a pulse sequence in a timeperiod before the specified imaging time, with regard to each pixel of aplurality of pixels;

calculating a pixel value of the pixel according to the pulse sequence;

obtaining an image at the specified imaging time according to a spacearrangement of the pixels, in accordance with pixel values of theplurality of pixels at the specified imaging time.

In an embodiment, a pulse signal-based display system is furtherprovided. The system includes: a pulse analysis module, a dynamicadjustment module and an image display module connected in sequence.

The pulse analysis module is configured to analyze pulse data to obtainpulse-firing information corresponding to individual pulse sequences ofindividual pixel positions.

The dynamic adjustment module is configured to determine a pixelthreshold range based on the pulse-firing information, and calculatepixel values of the pixel positions.

The image display module is configured to generate an image and outputthe generated image.

In the pulse signal-based display method according to the presentdisclosure, pulse-firing information of pulse sequences is obtained byanalyzing pulse data. Pixel values are calculated based on thepulse-firing information. Pulse signals can be visually displayed basedon a biological neuron pulse-firing mechanism. In this way, the timedomain characteristic of the pulse signal is effectively utilized, animage with high quality is formed and an image at any continuous timeinstant is output. Firing information (firing frequency) in the pulsesignal is calculated and a statistic can be made. Pixel values during anaccumulated time duration of pixel positions and a global pixel value ofthe entire image is calculated, thereby determining a pixel thresholdrange of each pixel position. In this way, pixel thresholds can berespectively set for pixels at the pixel positions based on temporal andspatial relationships, so that the pixel thresholds of the respectivepixel positions are relatively independent, thereby effectivelyimproving the quality of an output image. In a case where a differencebetween the first pixel and the pixel threshold range is small, thepixel threshold range is adjusted to include the first pixel value,which is advantageous for quality improvement of finally generatedimage. An unclearness caused by over-bright image or over-dark image canbe reduced by adjusting the pixel values of the finally generated imagebased on the pixel threshold range. By calculating the pixel values inthe accumulated time durations of the pixel positions and dynamicallyadjusting the first specific amount, the processing on the pixels insubsequent steps can be facilitated. The method is applicable to data invarious pulse forms, including the generation of color images. Thegenerated image can be used as a data source for machine learningalgorithms and other image and video recognition algorithms.

Those of ordinary skill in the art can understand that the drawings arejust schematic diagrams of an embodiment, and the modules or processesin the drawings are not necessarily required to implement the presentdisclosure.

It can be known from description of the above embodiments that, thoseskilled in the art may clearly understand that the present disclosuremay be implemented by means of software in combination with a necessarygeneral hardware platform. Based on such understanding, the essentialpart or the part contributing to the prior art of technical solutions ofthe present disclosure may be embodied in the form of a softwareproduct. The computer software produce may be stored in a storage mediumsuch as a ROM/RAM, a magnetic disc, an optical disc, etc., includingseveral instructions to enable a computer device (which may be apersonal computer, a server, or a network device) to execute the methodsdescribed in the embodiments of the present disclosure or some parts ofthe embodiments.

The embodiments in this specification are described in a progressivemanner. The identical or similar parts among the embodiments can bereferred to each other. Each embodiment focuses on differences fromother embodiments. In particular, for the device or system embodiments,since they are basically similar to the method embodiments, thedescription is relatively simple, and the relevant parts may be obtainedwith reference to the description of the corresponding parts of themethod embodiments. Device and system embodiments described above aremerely exemplary, in which the units described as separate parts may ormay not be physically separate, and the parts displayed as units may ormay not be physical units, that is, they may be located in one place, ormay be distributed to a plurality of network units. Some or all of themodules may be selected according to actual needs to achieve thepurposes of the solutions of the embodiments. Those of ordinary skill inthe art may understand and implement the present disclosure withoutcreative efforts.

The above descriptions are merely preferred specific embodiments of thepresent disclosure, but the protection scope of the present disclosureis not limited thereto. Variations or alternatives that may be easilyderived by those skilled in the art within the technical scope disclosedby the present disclosure should fall in the protection scope of thepresent disclosure. Therefore, the protection scope of the presentdisclosure shall be based on the protection scope of the claims.

What is claimed is:
 1. An imaging method, comprising: acquiring, at aspecified imaging time, a pulse sequence in a time period before thespecified imaging time, with regard to each pixel of a plurality ofpixels; calculating a pixel value of the pixel according to the pulsesequence; obtaining an image at the specified imaging time according toa space arrangement of the pixels, in accordance with pixel values ofthe plurality of pixels at the specified imaging time.
 2. The imagingmethod according to claim 1, wherein the calculating a pixel value ofthe pixel according to the pulse sequence comprises: calculating aquantity of pulses in a fixed period of time before the specifiedimaging time; obtaining the pixel value according to the quantity ofpulses.
 3. The imaging method according to claim 1, wherein thecalculating a pixel value of the pixel according to the pulse sequencecomprises: acquiring a length of time interval between adjacent pulsesin the pulse sequence, calculating an intensity value corresponding tothe time interval according to the length of the time interval, andobtaining a first accumulated pixel value according to one or moreintensity values; wherein the obtaining an image at the specifiedimaging time according to a space arrangement of the pixels, inaccordance with pixel values of the plurality of pixels at the specifiedimaging time comprises: obtaining the image at the specified imagingtime according to the first accumulated pixel value at a pixel positioncorresponding to each pixel of the plurality of pixels.
 4. The imagingmethod according to claim 3, wherein the acquiring a length of timeinterval between adjacent pulses in the pulse sequence, calculating anintensity value corresponding to the time interval according to thelength of the time interval, and obtaining a first accumulated pixelvalue according to one or more intensity values comprises: accumulatingat least one intensity value before the specified imaging time to obtainthe first accumulating pixel value.
 5. The imaging method according toclaim 3, wherein the obtaining an image at the specified imaging timeaccording to a space arrangement of the pixels, in accordance with pixelvalues of the plurality of pixels at the specified imaging timecomprises: setting a first specific amount corresponding to thespecified imaging time of the pixel position, and summing the firstspecific amount and the first accumulated pixel value to obtain a firstpixel value of the pixel position; and obtaining the image at thespecified imaging time according to the first specific amount at pixelpositions corresponding to the plurality of pixels.
 6. The imagingmethod according to claim 5, wherein the obtaining an image at thespecified imaging time according to a space arrangement of the pixels,in accordance with pixel values of the plurality of pixels at thespecified imaging time comprises: comparing the first pixel value with apixel threshold range, and obtaining a second specific amount byadjusting the first specific amount; summing the first accumulated pixelvalue and the second specific amount to obtain a second pixel value ofthe pixel position; and obtaining the image at the specified imagingtime according to the second pixel value at pixel positionscorresponding to the plurality of pixels.
 7. The imaging methodaccording to claim 3, further comprising: determining an accumulatedtime duration before the specified imaging time; wherein the acquiring alength of time interval between adjacent pulses in the pulse sequence,calculating an intensity value corresponding to the time intervalaccording to the length of the time interval, and obtaining a firstaccumulated pixel value according to one or more intensity values,comprises: acquiring at least one intensity value corresponding to atleast one time interval during the accumulated time duration;calculating at least one attenuation value of the at least one intensityvalue after it is attenuated when reaching the specified imaging time;summing the at least one attenuation value to obtain the firstaccumulated pixel value.
 8. The imaging method according to claim 7,further comprising: setting the first accumulated pixel value at 0 ifthere is no fired pulse in the accumulated time duration.
 9. The imagingmethod according to claim 5, wherein an input pulse signal at the pixelposition can change first accumulated pixel values or first specificamounts at one or more other pixel positions.
 10. The imaging methodaccording to claim 6, wherein a first pixel value at the pixel positioncan change second specific amounts or pixel values at one or more otherpixel positions.
 11. The imaging method according to claim 6, whereinthe comparing the first pixel value with a pixel threshold range, andobtaining a second specific amount by adjusting the first specificamount comprises: obtaining the second specific amount by adjusting thevalue of the first specific amount if the first pixel value is notwithin the pixel threshold range; and determining the first specificamount as the second specific amount if the first pixel value is withinthe pixel threshold range.
 12. The imaging method according to claim 6,wherein the pixel threshold range is set as a fixed value.
 13. Theimaging method according to claim 6, wherein the pixel threshold rangeis determined based on normalized global pixel values.
 14. The imagingmethod according to claim 6, wherein the pixel threshold range isdetermined based on an ideal dynamic range of image.
 15. The imagingmethod according claim 5, wherein the setting the first specific amountcomprises: setting the first specific amount as a fixed value.
 16. Theimaging method according to claim 7, wherein the at least one intensityvalue is a function in which the at least one pixel value is attenuatedat a fixed ratio with the time interval.
 17. The imaging methodaccording to claim 7, wherein the at least one intensity value is o afunction in which the at least one pixel value is attenuated at a fixedmagnitude with the time interval.
 18. The imaging method according toclaim 7, wherein the at least one pixel value is a function in which theat least one pixel value is attenuated by being decreased with adecreasing part of the function with the time interval.
 19. The imagingmethod according to claim 6, wherein the obtaining the image at thesingle pulse-firing time according to the second pixel value at pixelpositions corresponding to the plurality of pixels comprises: filteringthe second pixel value based on a temporal neighbor relationship of thepixel values at the pixel position; obtaining the image at the specifiedimaging time according to the filtered second pixel value.
 20. Theimaging method according to claim 6, wherein the obtaining the image atthe single pulse-firing time according to the second pixel value atpixel positions corresponding to the plurality of pixels comprises:filtering the second pixel value based on a spatial neighborrelationship of the pixel values at the pixel position; obtaining theimage at the specified imaging time according to the filtered secondpixel value.
 21. An imaging device, comprising: at least one processor;and at least one memory including computer program code, the at leastone memory and the computer program code configured to, with the atleast one processor, cause the device to perform at least the following:acquiring, at a specified imaging time, a pulse sequence in a timeperiod before the specified imaging time, with regard to each pixel of aplurality of pixels; calculating a pixel value of the pixel according tothe pulse sequence; obtaining an image at the specified imaging timeaccording to a space arrangement of the pixels, in accordance with pixelvalues of the plurality of pixels at the specified imaging time.